1 link tagged with all of: interpretability + model editing + mit csail + astra + weight space programming
Click any tag below to further narrow down your results
Links
MIT CSAIL researchers had an AI model inspect and patch the numerical weights of a small open Qwen model, raising its MATH-500 score from 7.9% to 48.1% by touching about 0.016% of them. The gain came mostly from removing a repetition problem, not from adding new math ability.
- The edit changed only 131,072 weights in layers 15 and 19, cutting responses that hit the token limit from 93% to 33%.
- Finding that patch was expensive: the strongest run took 20.3 hours, 80 Python runs and 50 evaluation requests, including a 4.7-hour outage wait.
- A control on Gemma 3 270M showed no math gain, which suggests the method repairs flaws rather than adding knowledge.
- Critics split between doubting the reasoning claims and warning that the same tools could speed up capability development.